Go Core WG Roadmap #4
Conversation
daviddias
assigned eingenito, Stebalien, michaelavila, hannahhoward, schomatis, kevina, Kubuxu, djdv, keks, momack2 and magik6k
|
Hi Go Core Team! Now that the Roadmap is a live PR on Github, it is a good time to check in on our Roadmap Instruction Manuals and check in to see if the Roadmap contemplates everything we could have wished for and everything we believe it is valuable for the community to have access. Think things like: Is the Vision described? Are the Requirements well understood? We have a good sense of priority and dependencies? You can find the Roadmap Instruction Manuals at:
Check in with your WG and have fun giving this review! :) |
daviddias
force-pushed
the
roadmap-wg-go-core
branch
from
49b7396
to
bc48a4b
Compare
ghost
added
the
| * `M (P2)`: (machine-learning/package managers) IPFS can handle (and transfer) large (>1M entries) sharded indexes (objects, directories) | ||
| * Bitswap Improvements (sessions, prediction, etc) | ||
| * UnixFS-V2 (better directory structure) | ||
| * `M (P3)`: (Dapps) IPFS can locally share data without a shared network |
There was a problem hiding this comment.
@mgoelzer @raulk - FYI a bluetooth(like) transport for locally sharing data without a shared network is a requested item in our go-ipfs 2019 roadmap
There was a problem hiding this comment.
Hey, thanks for pinging me on this. I'm really interested in the BT transport (and Wifi), but I'm not sure who would work on it. From past conversations with @whyrusleeping I believe it requires someone who has done low-level Bluetooth development before, and there is no one at PL (afaik) who falls in this category. We discussed this at the Lisbon Hack Week in May 2018 and I believe @mafintosh (think it was him?) said he knew someone with this experience, but we never followed up on it. @whyrusleeping also has done some research on this in the past, but lately he's been focusing on other projects.
If 2019 is the year that we get serious about a BT transport in libp2p, I think we should put our heads together and (1) create an RFP with acceptance criteria for exactly what we want, and (2) think of who we know who might be able to take this on.
There was a problem hiding this comment.
Added a crosslink in the libp2p roadmap: https://docs.google.com/document/d/1Rd4yNw1TNQBvfRrKeEMSTseb6fvPzS-C--obOn0nul8/edit?disco=AAAACYoDclo We need to get better at tracking these crosslinks in a visual manner.
| * `M (P0)`: ipfs://, ipns:// work in web browsers | ||
| * Base32 CIDs by default | ||
| * Base32 IPNS keys (or make IPNS keys CIDs) | ||
| * `M (PX)`: <1 sec IPNS resolution for any IPNS record |
There was a problem hiding this comment.
| * `M (PX)`: <1 sec IPNS resolution for any IPNS record | |
| * `M (PX)`: <5 sec IPNS resolution for any IPNS record |
There was a problem hiding this comment.
Although I'm a bit worried that this just isn't viable. That is, 5s to resolve something is really slow (but we're not going to get much better on our DHT). @lgierth what's the status of your dns-based IPNS system?
| * NAT detection/traversal | ||
| * Only "reachable" nodes join the DHT | ||
| * `M (P1)`: IPFS provides a clean way to persist data (dapps need to store stuff) | ||
| * `M (PX)`: IPFS supports basic dapp requirements for security and private content |
There was a problem hiding this comment.
I don't think we're going to get to this until Q2 at the earliest, but it'd be nice to take a pass on this work before doing a security audit to make that analysis more valuable. So maybe API security in Q2 and examples/documentation in Q3?
| * `M (PX)`: IPFS supports basic dapp requirements for security and private content | |
| * `M (PX)`: IPFS supports basic dapp requirements for security and private content | |
| * Q2 - dapps are secure (api security, multi tenancy). Apps can't read each other's state | |
| * Q3 - There are examples and documentation for ipfs users to create private-content dapps using IPFS (aka handling encryption on the client side) |
| * `M (P2)`: (machine-learning/package managers) IPFS can handle (and transfer) large (>1M entries) sharded indexes (objects, directories) | ||
| * Bitswap Improvements (sessions, prediction, etc) | ||
| * UnixFS-V2 (better directory structure) | ||
| * `M (P3)`: (Dapps) IPFS can locally share data without a shared network |
There was a problem hiding this comment.
Creating an RFP in Q1 would allow us to find folks interested in working on this in Q2, allowing others to build on this capability in H2.
| * `M (P3)`: (Dapps) IPFS can locally share data without a shared network | |
| * `M (P3)`: (Dapps) IPFS can locally share data without a shared network | |
| * Q1 - Draft RFP for bluetooth (or like) transport and find owner | |
| * Q2 - Implement bluetooth (or like) transport |
| * UnixFS-V2 (better directory structure) | ||
| * `M (P3)`: (Dapps) IPFS can locally share data without a shared network | ||
| * A bluetooth (or like) transport | ||
| * `M (P4)`: (anti-censorship) Our IPFS implementations are secure and don't leak sensitive information |
There was a problem hiding this comment.
Ideally we do a security pass as a team prior to a 3rd party security audit, and then have explicit time for fixes and documentation. Question - should we wait on privacy-preserving efforts until after we have audited the security of the core of IPFS, or should we have some experimental privacy-preserving efforts that are evaluated as part of our security audit?
Q1 - find 3rd security auditors
Q2 - Implement api security, multi tenancy
End of Q2 - Security audit
Q3 - Implement fixes based on security audit
Q3 - document how to use IPFS securely
?? - Privacy-preserving DHT / Transport
There was a problem hiding this comment.
We can start those efforts but I'm not sure if it's worth including them in a security audit (ever).
| * Graphsync selector implementation that can understand non-overlapping subsets of data to select from multiple hosts and the driver that can build up graphsync requests for many peers to maximize efficiency and throughput. | ||
| * `M (P0)`: Go-IPFS can transfer many small files at 0.8 times the speed of rsync (for single peers) or BitTorrent (for many peers) | ||
| * Bitswap session improvements | ||
| * Graph Exchange that can route requests efficiently |
There was a problem hiding this comment.
| * Graph Exchange that can route requests efficiently | |
| * Q1 - Graph Exchange that can route requests efficiently |
| * `M (P1)`: Go-IPFS can add a directory of many small files at 50% the speed of a simple local hard drive copy (or dd) (10k x 1MB files) | ||
| * Flexible provider strategies to limit the number of DHT provider advertisements we make | ||
| * Faster datastore (Badger or equivalent) that doesn’t slow down as the number of blocks stored grows large | ||
| * GC improvements (transactions, speed) |
There was a problem hiding this comment.
| * GC improvements (transactions, speed) | |
| * Q2 - GC improvements (transactions, speed) |
It would certainly be nice if this was Q1, but I'm going to guess that Q2 is more likely?
Marking Q1 initiatives and adding them to Q1 OKRs in https://github.com/ipfs/team-mgmt/pull/794/files Co-Authored-By: momack2 <momack2@users.noreply.github.com>
|
I'm not sure which of the existing lines exactly to attach this to, so, I'll just make it as a general comment -- "UnixFS-v2" shows up in roughly three places; IPLD shows up in roughly zero (except nonspecifically, at the top). I'd like to put 'IPLD schemas' in here somewhere. Since there's actually considerable progress on that front recently, I'd like to double-down on it and actually declare this is what we're looking for to e.g. drive better APIs for making Dapps. And UnixFS-v2 should actually be re-hung as a dependency of IPLD Schemas; we were cautious of that before, because schemas seemed far away, but now that they're looking closer, I think we should seriously consider actually making them a dependency and forcing ourselves to ship it -- otherwise, we're just going to end up with a bullet point in the not-so-distant future that's a wishlist for a UnixFS-v3 which is built with native specification as IPLD schema. |
|
Merging this PR to document where we are on our roadmaps - future iterations to narrow down on a more focused and achievable set of roadmap initiatives (to match project-level priority update) will follow in next few weeks in a separate PR. |
Preview: https://github.com/ipfs/roadmap/blob/roadmap-wg-go-core/WG_GO_CORE.md